MASSACHUSETTS  INST  OF  TECH  CAMBRIDGE  LAB  FOR  INFORMA— ETC  F/6  17/2 
AN  OPTIMAL  CONTROL  APPROACH  TO  DYNAMIC  ROUTING  IN  NETWORKS*  PAR— ETCCUI 
JUN81  M  JOOORKOVSKY  #  A  SEGALL  N0001H-75-C-1183 

UNCLASSIFIED  LIDS-P-UOO  NL 


SECURITY  CLASSIFICATION  of  This  page  (When  Data  Entered) 


REPORT  DOCUMENTATION  PAGE 


4.  TITLE  (end  Subtitle) 


READ  INSTRUCTIONS 
BEFORE  COMPLETING  FORM 


ECl  Pi  EN  T*S  CATALOG  NUMBER 


S.  TYPE  OF  REPORT  *  PERIOD  COVERED 


4an  .OPTIMAL  qontrol  j^pproach  to  dynamic  routing 

'in  .NETWORKS.”  PART  XI .  A  MAXIMAL  JLOW  £PPROACH 


\)  Mario*)jodorkovsky 
Adrian ^Segall  ; — 


Paper 


9.  PERFORMING  ORGANIZATION  NAME  ANO  ADDRESS  10-  PROGRAM  ELEMENT  PROJECT.  TASK 

. ,  «  _  .  pm  _  •  ^  Ant  A  O  UNI  I  NUMDt 

Massachusetts  Institute  of  Technology  Program  Code  No.  5T10 

Laboratory  for  Information  and  Decision  Systems  QNR  Identif  ing  No.049-383 
Cambridge,  MA  02139 


II.  CONTROLLING  OFFICE  NAME  ANO  ADDRESS  ' 

Defense  Advanced  Research  Projects  Agency  jJ - 

1400  Wilson  Boulevard 

Arlington,  Virginia  22209 _  . 

14.  MONITORING  AGENCY  NAME  4  ADDRESS  (it  ditfaranl  from  Controlling  Otflca)  I  IS.  SECURITY  CLASS,  (ol  thia  raport) 


13.  NUMBER  OF  PAGES 


Office  of  Naval  Research 
Information  Systems  Program 
Code  437 

Arlington,  Virginia  22217 


IS.  DISTRIBUTION  STATEMENT  (oi  thia  RaporlJ 


Approved  for  public  release;  distribution  unlimited 


17.  DISTRIBUTION  STATEMENT  (ol  tha  abatract  antarad  in  Block  20.  ft  dl/farent  tram  Repo, 


18.  SUPPLEMENTARY  NOTES 


Unclassified 


IS*.  DECLASSIFICATION/  DOWNGRADING 
SCHEDULE 


/DT1C 

ELECT'D 
JUL  2  1 1981 


This  paper  has  been  submitted  to  the  IEEE  Transactions  on  Automatic  Control. 


19.  KEY  WORDS  (Continue  on  reverse  side  it  necessary  and  identity  by  block  number) 


20.  ABSTRACT  (Continue  on  reverse  side  If  necessary  and  identify  by  block  number) 

This  paper  presents  a  new  approach  for  building  the  feedback  solution 
for  the  minimum  delay  dynamic  message  routing  problem  for  single  destination 
networks.  The  necessary  and  sufficient  conditions  for  optimality  derived 
in  previous  works  are  interpreted  in  terms  of  weighted  maximal  flow  problems. 
Several  properties  concerning  these  problems  are  obtained  and  used  to  develop 
a  simplified  algorithm  for  building  the  feedback  space.  The  various  steps 
of  the  algorithm  are  presented  and  motivated  in  detail. 


,  J  AN  73  1473  EOITION  OF  I  NOV  85  IS  OBSOLETE 


cf  i  c  r 


SECURITY  CLASSIFICATION  OF  THIS  PAGE  (l*7i»n  D»f*  Entrrad) 


Jk  i 


June,  1981 


LIDS-P-1100 


AN  OPTIMAL  CONTROL  APPROACH  TO  DYNAMIC  ROUTING  IN  NETWORKS 
PART  II:  A  MAXIMAL  FLOW  APPROACH 


Mario  Jodorkovsky  and  Adrian  Segall 
Department  of  Electrical  Engineering 
Technion  -  Israel  Institute  of  Technology 
Haifa,  Israel 


ABSTRACT 

.'i 

This  paper  presents  a  new  approach  for  building  the  feedback  solution 
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I .  INTRODUCTION 


In  previous  papers  [1],  [2],  a  model  for  the  analysis  of  dynamic  routing 

in  networks  has  been  proposed  and  the  principles  of  an  algorithm  for  the 

solution  of  the  resulting  dynamic  equations  have  been  introduced.  The  model 

gives  rise  to  a  linear  optimal  control  problem  with  linear  state  and  control 

inequality  constraints  and  linear  integral  cost  functional.  The  application 

of  the  maximum  principle  to  this  model  dictates  that  the  necessary  and 

sufficient  condition  for  a  control  law  to  be  optimal  is  to  be  the  solution  of  a 

% 

certain  linear  program  parametrized  by  time.  Furthermore,  it  is  indicated  in 
[2]  that  the  conceptual  algorithm  for  the  solution  of  the  dynamic  problem  can 
be  made  implementable  for  the  case  when  all  traffic  in  the  network  has  a 
single  destination  and  there  are  no  priorities,  and  in  this  papeT  we  restrict 
our  attention  to  this  class  of  problems. 

The  main  purpose  of  the  present  paper  is  twofold.  First  we  int-roduce  a 
new  interpretation  to  the  (parametrized)  linear  program  that  provides  the 
necessary  and  sufficient  condition  for  optimality  in  terms  of  a  weighted 
maximal  flow  problem.  It  is  well  known  that  algorithms  for  maximal  flow 
problems  are  much  more  efficient  than  those  for  general  linear  programs,  so 
that  this  interpretation  may  offer  large  computation  savings.  In  our  case, 
this  point  o/  view  provides  the  additional  benefit  of  allowing  us  to  obtain 
a  series  of  properties  with  extensive  implications  to  the  development  of  the 
actual  algorithm.  The  introduction  of  this  maximal  flow  approach  is  the 
subject  of  Section  II. 

The  second  purpose  of  the  present  paper  is  to  present  an  algorithm  for 
the  solution  of  the  optimal  control  problem.  The  insight  and  properties  of 
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the  maximal  flow  approach  developed  in  Section  II  together  with  the  main 
properties  of  the  problem  in  [2,  Theorem  4]  allow  us  to  considerably  reduce 
the  complexity  of  the  algorithm,  converting  it  into  an  implementable  algorithm 
for  moderate  size  networks.  In  Section  III  the  algorithm  is  introduced, 
motivated  and  applied  to  an  illustrative  example,  and  several  concluding 
remarks  appear  in  Section  IV. 

i 

II.  THE  MAXIMAL  FLOW  APPROACH 

As  said  before,  we  restrict  our  attention  to  the  case  of  single  destina¬ 
tion  networks  with  unity  weightings  in  the  cost  functional.  In  addition,  we 
assume  that  all  the  input  rates  to  the  network  are  zero,  so  that  our  problem 
is  to  find  the  optimal  control  (routing)  that  empties  the  network,  starting 
from  a  given  distribution  of  backlogged  traffic.  Theorem  4  in  [2]  gives 
the  main  properties  of  the  solution  and  in  the  present  paper  we  heavily  rely 
on  these  results,  as  well  as  on  the  main  optimality  conditions  obtained  in 
[2,  Theorems  1,2]. 

The  dynamic  equations,  constraints  and  cost  functional  for  the  problem 
are  (2,  Eq.  (7)-(ll)]: 

» 

x(t)  »  B  u(t)  ;  x(tQ)  *  xQ  ;  x(tf)  =  0  ;  x(t)  >  0  (1) 

U  «  {u(t)  <  C  ;  u(t)  >  0}  (2) 

J  -  j  Ix.(t)dt  ,  (3) 

o 


where  B  is  the  incidence  mattix  of  the  network  and  C  the  capacity  vector. 


Necessary  and  sufficient  conditions  for  a  control  law  u*(t)  to  be  optimal 
is  to  satisfy  (1)  and  also  to  solve  the  linear  program 

■in  [XT(t)x(t)],  all  t  €  It  ,tf],  (4) 

u(t)€U  " 

where  X(t)  is  a  costate  vector  that  satisfies  the  appropriate  backward 
dynamic  equations  [2,  Eq.  (15)-(19)].  In  view  of  [2,  Thai.  4d)],  there  is  no 
loss  of  generality  if  we  restrict  our  attention  to  trajectories  with  non¬ 
positive  slopes  (forward  in  time),  so  that  (4)  can  be  rewritten 

* 

* 

min(XTxJ  ,  u  €  U,  x  <  0  (5) 

where,  for  simplicity  of  notation,  we  have  also  suppressed  explicit  reference 
to  time,  A  control  vector  u*  satisfying  (1)  and  (5)  is  said  to  be  globally 
optimal. 

For  a  given  time  t  and  a  given  trajectory  x(»).  let  B  and  I  dmtote 
the  sets  of  states  Xj  for  which  x^t)  •  0  and  x^t)  >  0  respectively 
(named  boundary  and  interior  sets).  A  control  vector  satisfying  (1)  and 

*in[X^x]  ,  u  €  U,  x^  <  0  for  x^  €  I  (6) 

x^  *  0  foT  Xj  €  B 

is  said  to 'be  constrained  optimal  [2,  Note  1).  Clearly,  every  global  optimal 
control  is  constrained  optimal  and  [2,  Tha.  4c)  states  that  for  the  case 
under  consideration  the  opposite  is  true  as  well.  Consequently,  we  can  regard 
(6)  as  replacing  (5)  in  the  necessary  and  sufficient  conditions  for  optimality, 
and  the  rest  of  the  present  section  will  be  devoted  mainly  to  the  analysis  of 
equation  (6). 
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First  observe  that  by  defining  y^  *  -x^  for  €  I  and  by  explicitly 
writing  the  incidence  matrix  B  and  the  vector  u  (see  [2,  Eq.  (1)]),  the 
minimization  problem  (6)  becomes: 

max  F  =  2  X.y.  (7a) 

•  .  —  C  T  1  1 


with  constraints 

-  2  u.  .  +  2  u.  .  *  y.  =0 
j  «  k  ki  1 

Vi :  xi  €  I 

(7b) 

- z  uij  *  *  uu  * 0 

3  * 

Vi:  x.  €  3 

i 

(7c) 

y.  >  0,  u  €  U. 

(7d) 

Now  observe  that  problem  (7)  is  a  (weighted)  maximal  flow  problem  for  a  net¬ 
work  that  is  obtained  from  the  original  network  by  adding  an  extra  node, 
named  source  s,  and  links  with  infinite  capacity  connecting  s  to  all  nodes 
i  for  which  x.  ?  I  (see  Fig.  1).  The  network  of  Fig.  lb)  will  be  called 
the  network  corresponding  to  problem  (7)  or  the  network  corresponding  to  the 
set  I.  The  main  idea  of  the  present  work  is  to  exploit  properties  of  this 
maximal  flow  problem  and  algorithms  for  its  solution,  in  order  to  obtain 
properties  and  algorithms  for  the  solution  of  our  original  dynamic  control 
problem.  The'rest  of  this  section  is  devoted  to  the  exposition  of  these 
properties. 

Theorem  1: 

(a)  The  costates  X^(t)  corresponding  to  states  x..  (t)  €  I  are  strictly 
positive. 

(b)  Any  optimal  solution  of  problem  (7)  with  arbitrary  positive  X's  is 
also  an  optimal  solution  of  problem  (7)  with  X^  •  1  Vx^  6  J. 


/ 
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Proof:  a)  Assume  that  x^  reaches  the  boundary  (forward  in  time)  at  time 
T  with  an  optimal  slope  x^  <  0.  Now  clearly  Ai (t)  >  0,  since  otherwise 
the  necessary  and  sufficient  condition  (4)  says  that  it  is  not  optimal  for 
x^  to  reach  the  boundary  at  x.  Moreover,  the  dynamic  equation  for  costates 
corresponding  to  states  travelling  on  interior  arcs  [2,  Eq.  (IS) -(17)]  is 
-X^(t)  »  1  and  therefore  X^(t)  >  0  for  all  x^(t)  €  I. 

b)  Let  be  the  optimal  value  of  problem  (7)  with  X^  ■  1  for  all 

x.  €  I  and  let  y*  be  a  solution  vector  of  problem  (7)  with  arbitrary 

« 

X.  >  0  for  x.  €  J.  Clearly  2  y*  <  F,  and,  on  the  other  hand,  the 
1  1  x.€J  1  1 

inequality  cannot  be  strict  since1  otherwise,  the  Maximal  Flow  Theorem  [3] 

implies  that  a  path  can  be  found  between  the  source  and  destination  nodes 

of  the  network  corresponding  to  problem  (7)  on  which  the  flow  can  be 

increased,  contradicting  the  fact  that  y*  solves  (7).  Consequently, 

Z  y*  -  F  .  (8) 

XiCI 

implying  that  y7  is  a  solution  of  (7)  with  XA  *  1  Vx^  €  I. 

From  now  on,  and  as  a  result  of  Theorem  1  we  concentrate  on  problem  (7) 
with  X^  *  1,  Vx^  €  J,  and  use  the  notations  N[I)  for  the  network  correspond¬ 
ing  to  the  set  J,  MFP(I)  for  the  maximal  flow  problem  (7)  corresponding  to 
the  set  I  with  X^  *  1,  Vx^  6  I  and  max(J)  for  the  maximal  flow  value 
corresponding  to  MFP(J) . 

Theorem  2: 

(a)  Suppose  there  exists  two  sets  of  states  J  and  J’  such  that  J  <=  I’ 
and  max (I)  *  max(I').  Then  all  basic  optimal  solutions  of  MFP(I)  are 
basic  optimal  solutions  of  MFP(I'),  and  all  minimal  cuts  of  N(I) 
are  also  minimal  cuts  of  AP(I'). 


b)  For  a  given  set  J,  if  there  is  no  set  I '  satisfying  the  conditions  in 
(a),  then  a  minimal  cut  of  H (J)  is  (X,X),  where: 

X  *  {s}  U  {Xi  €  J}. 

Moreover,  when  max(J)  is  achieved,  then  all  links 
{(i,k)  €  N(I)  |  x.  £  J,  xk  €  J) 

have  flow  equal  to  the  corresponding  capacity  cik  and  all  links 
{(j,i)  6  N(D  |  x.  €  J,  x.  C  I } 
have  zero  flow. 

c)  If  there  exist  two  sets  I  and  I'  such  that  I  c  I’  and 

max(J)  »  max(J'),  then  for  any  set  J”  holds  max(J  U  I”)  *  maxfJ'UJ") 

Proof:  a)  Take  all  basic  optimal  solutions  of  MFP(J)  and  add  new  variables 
{y±  >  0,  Vx.  €  I’/J)  as  non-basic  variables  with  value  zero.  Clearly, 
all  these  solutions  satisfy  the  constraints  of  MFP(J').  Moreover,  since 
max  J =  maxJf  and  the  new  variables  are  non-basic,  the  solutions  are  also 
basic  optimal  solutions  for  MFP(If).  Take  now  all  solutions  of  MFP(I') 
satisfying  {y^  =  0,  Vx^  £  I' /I }.  Removing  from  #(Jf)  the  links  correspond 
ing  to  {y^,  VXj  £  I'/l)  will  give  exactly  N{T)  hence  all  minimal  cuts 
of  N(I)  are  minimal  cuts  of  JV(I'). 

b)  Clearly,  every  node  {i  {  x^  £  1}  belongs  to  X  since  there  are  no  upper 
bounds  on  the  variables  y^.  Now,  suppose  that  there  exists  a  minimal  cut 
(X’.X’)  such  that  X’  ■  XUj  where  x^  C  I.  Then  we  can  add  to  the  network 
H{T)  a  new  variable  y .  >  0  as  non-basic  with  value  zero  (i.e.  a  link 
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connecting  the  source  with  node  j  with  zero  flow).  But  note  that 
I  e  (J  U  Xj)  and  max(J)  *  max(I  U  x^)  contradicting  the  assumptions. 

By  the  Maximal- FI ow-Min-Cut  Theorem  [3]  and  links  connecting  X  with  X 
have  flow  equal  to  their  capacity  and  all  links  connecting  X  with  X 
have  no  flow. 

c)  Consider  a  solution  of  MFP(J')  satisfying  (y^  =  0,  Vx^  €  I' /I).  Part 
fa)  ensures  the  existence  of  such  a  solution.  Now  add  to  the  network  N(I') 
links  corresponding  to  {y^  |  x^  €  In ,  x^  C  I'  H  J")  and  achieve  maximal 
flow  in  the  resulting  network  N(In  U  J')  without  changing  the  flow  on  links 
{/i  |  Xi  €  J'}.  Now  remove  the  links  {y^  |  x^  £  I' /I,  C  I").  Since 
those  links  have  no  flow,  the  maximal  flow  does  not  change,  therefore 
max(I  UI")  =  max(I'  U  I"). 

The  proof  of  the  next  theorem  is  somewhat  long  and  therefore  not 
included  here: 

Theorem  3:  [4,  Thrms.  3.1,  3.2]  Assume  that  the  costates  corresponding 

to  states  on  the  boundary  are  kept  at  their  leave- the-boundary  value  (cf. 

[2,  Thm.  4a])  at  all  times.  Then  all  costates  satisfy  A^(t)  =0  or  -1  and 

Vt)  -  X.(t)  *  0 

if  and  only  if  the  following  hold  : 

(i)  x.(t)  £  B 

(ii)  there  are  no  sets  I*  and  I  such  that  x^  €  J',  x^  C  J,  Icj', 
maxi*  max/'  and  all  states  in  I  have  left  the  boundary  before 
time  t  backwards  in  time  (i.e.  at  some  time  strictly  larger  than  t). 
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III.  THE  ALGORITHM  FOR  BUILDING  THE  FEEDBACK  CONTROL  REGIONS 

With  the  results  of  Section  II,  we  are  ready  to  introduce  the  algorithm 
that  implements  the  constructive  dynamic  programming  concept  introduced  in 
[2]  and  builds  the  feedback  control  regions  that  represent  the  solution  to 
the  optimal  dynamic  routing  problem.  In  order  to  obtain  a  complete  character¬ 
ization  of  the  feedback  space  one  has  in  principle  to  find  all  backward 
optimal  trajectories  and  the  corresponding  optimal  control  vectors.  However, 
due  to  the  special  nature  of  the  problem,  the  results  of  [2]  and  the  properties 
of  Section  II  of  this  paper,  we  can  obtain  an  algorithm  that  uses  only  a  finite 
number  of  trajectories  (as  opposed  to  all  optimal  trajectories).  Furthermore, 
the  same  results  allow  us  to  reduce  drastically  the  number  of  considered 
trajectories  as  well  as  the  number  and  complexity  of  the  operations  needed  to 
generate  the  trajectories  and  to  translate  them  into  the  feedback  control 
space.  The  result  is  that  we  obtain  an  implementable  algorithm,  at  least  for 
moderate  size  networks.  In  what  follows  we  first  intuitively  explain  the 
possible  reductions  and  then  present  the  algorithm  formally. 

The  first  simplification  is  obtained  by  observing  that  from  [2,  Thm.  4] 
it  is  enough  to  consider  piecewise  linear  trajectories  with  nonpositive 
slopes  (forward  in  time)  and  no  breaks  between  junction  points.  These  can 

m 

be  represented  as  trajectories  with  the  property  that  various  sets  of  state 
variables  leave  the  boundary  x^  =  0  (backwards  in  time),  never  return  to  it 
and  once  a  certain  set  of  states  leaves  the  boundary,  the  optimal  control 
at  the  leave-the-boundary  time  remains  optimal  until  t  =  -<*>  unless  other 
states  leave  the  boundary  subsequently  (backwards  in  time).  Whenever  a 
certain  set  of  state  variables  is  assigned  to  leave  the  boundary,  the 


corresponding  set  of  costates  that  allows  this  to  happen  optimally  must  be 
found  (and  is  unique  according  to  [2,  Thm.  4a])  and  all  solutions  of  the 
corresponding  problem  (7)  must  be  obtained.  The  set  of  all  these  solutions 
will  determine  the  corresponding  control  feedback  region  (see  [2,  Example  1] 
for  illustration).  Here,  however,  we  obtain  a  number  of  further  simplifica¬ 
tions:  first,  since  (7)  is  a  linear  program,  it  is  enough  to  find  only  its 
basic  optimal  solutions  and  obtain  their  convex  hull,  since  all  other  optimal 
solutions  are  convex  combinations  of  the  basic  ones;  second.  Theorem  1 
above  says  that  the  solutions  of  problems  (7)  with  1.  =  l,¥x.  6  I,  are 
sufficient  and  there  is  no  need  to  solve  (7)  for  arbitrary  X's;  third. 

Theorem  2  a)  above  further  reduces  the  number  of  problems  of  type  (7)  that 
need  to  be  considered  to  those  corresponding  to  sets  I  for  which  there  is 
no  set  I’  such  that  I  c  J*  and  max(J)  =■  max(Ir);  fourth,  the  fact  that 
(7)  with  X^  =  1  is  a  maximal  flow  problem  allows  us  to  use  maximal  flow 
algorithms,  that  are  in  general  much  more  efficient  than  general  linear 
programs. 

Next,  observe  that  [2,  Thm.  4b)  and  e)]  imply  that  the  particular 
instants  when  sets  of  variables  leave  the  boundary  (backwards  in  time)  are 
immaterial  and  all  trajectories  with  identical  sequences  of  states  leaving 
the  boundary  construct  the  same  feedback  control  region  (see  also  [2,  Example  1]). 
Consequently,  the  leave-the-boundary  times  can  be  chosen  arbitrarily,  provid¬ 
ing  a  further  simplification  of  the  algorithm.  In  addition,  we  shall  show  in 
Operation  3  of  the  Algorithm  that  Theorems  2c  and  3  above  provide  a  way  for 
identifying  certain  sequences  of  states  leaving  the  boundary  that  are  redund¬ 
ant  in  the  sense  that  they  generate  feedback  control  regions  that  can  be  built 
using  other  sequences.  Clearly,  the  redundant  sequences  need  not  be  considered, 


10  - 


providing  a  further  saving  in  the  algorithm.  Finally,  we  may  mention  that 
Theorem  2b)  will  allow  us  to  reduce  the  size  of  the  corresponding  networks 
N(I)  on  which  one  needs  to  work  in  Operation  2  of  the  Algorithm. 

In  what  follows  we  present  the  algorithm  that  builds  the  feedback 
control  regions.  Each  step  of  the  algorithm  is  presented,  motivated  and 
applied  to  an  example  network  for  illustration.  The  illustrative  network 
is  given  in  Figure  2. 

Operation  1:  For  each  possible  subset  I  of  state  variables,  construct 

the  corresponding  network  N(I)  and  solve  the  corresponding  maximal  flow 
problem  MFP(J)  to  obtain  a  first  basic  optimal  solution  and  the  maximal 
flow  value  max(I). 

Motivation.  As  explained  before,  in  order  to  obtain  a  complete  characteriza¬ 
tion  of  the  feedback  space  we  have,  in  principle,  to  consider  all  possible 
trajectories  represented  by  sets  of  states  leaving  the  boundary  [backwards 
in  time).  For  each  set  of  states  leaving  the  boundary,  the  corresponding 
costates  should  be  calculated  and  the  corresponding  feedback  control  region 
is  obtained  by  finding  all  solutions  tc  problem  (7).  In  Operation  1,  we 
consider  all  possible  sets  of  state  variables  (there  are  2n-l  such  sets) 

and  find  a  first  optimal  (extremal)  solution  to  problem  (7)  for  each  set, 

* 

as  well  as  the  corresponding  maximal  flow  value.  Note,  in  addition,  that 
by  Theorem  1  it  is  enough  to  find  the  solutions  to  problem  (7)  with  =  1, 

Vx.  €  I. 

Illustration:  The  networks  corresponding  to  each  subset  I  for  the  example 
network  of  Figure  2  and  their  corresponding  maximal  flow  appear  in  Figure  3. 
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Operation  2:  Identify  those  sets  I  of  state  variables  for  which  there 

is  no  set  I'  ol  such  that  max(J')  ■  max(J).  For  these  sets  I  use  the 

method  indicated  in  the  Appendix  to  find  all  optimal  basic  solutions  of  the 
corresponding  MFP(J). 

Motivation:  In  principle,  in  order  to  obtain  an  exhaustive  set  of  trajector¬ 

ies  as  required  by  the  Constructive  Dynamic  Programming  Concept  [2] ,  there 
is  need  to  generate  all  basic  optimal  solutions  to  all  MFP(J)  determined  in 
Operation  1.  Here,  however,  we  were  able  to  obtain  large  savings  in  computa- 
tion  due  to  two  results.  First,  obtaining  all  solutions  to  a  linear  program 
is  an  extremely  tedious  operation  even  for  small  dimensions  as  seen  from  the 
literature  [5]-[9],  On  the  other  hand,  for  the  particular  type  of  problems 
under  consideration  here  we  were  able  to  develop  a  Maximal -Flow-specific 

procedure  (see  Appendix)  that  turns  out  to  be  especially  efficient  for 
0 

degenerate  problems,  which  is  the  situation  in  our  case.  The  second  saving 

is  obtained  from  Theorem  2a),  indicating  that  it  is  sufficient  to  obtain  all 

basic  solutions  only  to  MFP(J)  corresponding  to  those  sets  I  for  which 

there  is  no  proper  overset  with  the  same  maximal  flow  value,  thereby  reducing 

the  number  of  problems  to  solve.  Although  we  have  no  analytic  expression 

for  the  amount  of  savings,  all  examples  that  we  treated  provided  a  great  deal 
0 

of  reduction.  Furthermore,  observe  that  Theorem  2b)  specifies  the  minimal 
cut  corresponding  to  each  MFP  and  that  only  the  subnetwork  above  the  cut  needs 
to  be  considered  when  finding  all  solutions,  since  pivoting  under  the  cut 
will  not  lead  to  new  solutions  for  y^,  x^  €  I 

Illustration:  In  the  example  of  Figure  2,  we  need  to  consider  only  the  problems 
corresponding  to  the  networks  (a),  (b)  and  (e)  of  Figure  3.  The  different 


solutions  are  given  in  Table  1.  Note  also  that  by 
Theorem  la)  the  basic  optimal  solutions  of  MFP^.x-j}  and  of  MFPCx^.Xj} 
are  also  basic  optimal  solutions  of  MFPte^^.x^}  and  those  of  MFP{x^}  and 
MFP{x2)  are  also  basic  optimal  solutions  of  MFPtx^.Xj}. 

Before  introducing  Operation  3  we  may  indicate  that  our  goal  is  to  use 
representative  trajectories  in  the  state  space  in  order  to  construct  feedback 
control  regions  that  cover  the  entire  state  space.  In  principle,  in  order  to 
ensure  complete  coverage,  one  needs  to  list  all  possible  types  of  trajector¬ 
ies  characterized  by  sequences  of  sets  of  states  leaving  the  boundary. 

The  number  Q(n)  of  possible  trajectories  for  a  network  with  n  states 
(n*l  nodes)  is  given  by  [10] 

n  r 

Q(n)  -  2  I  (-1)  1(T)  (r-i) n 

r*l  i-0  1 

which  turns  out  to  grow  as  nn.  The  computational  complexity  of  considering 
these  sequences  (in  Operation  3)  in  order  to  construct  feedback  control 
regions  is  one  of  the  factors  that  limits  the  size  of  the  network  that  can 
be  handled  by  the  algorithm  with  reasonable  computation  effort.  However, 
one  of  the  important  results  that  we  have  been  able  to  obtain  by  using  the 
Maximal  Flow  approach  is  to  develop  a  criterion  in  order  to  discover 
redundant  sequences  in  the  sense  that  the  feedback  control  regions  generated 
by  the  corresponding  trajectories  are  in  fact  contained  in  such  regions 
generated  by  another  trajectory.  The  above  criterion  allows  us  to  check 
redundancy  before  proceeding  to  Operation  4  and  since  a  large  number  of 
trajectories  usually  turn  out  to  be  redundant,  this  considerably  reduces  the 
computational  requirements  of  the  Algorithm. 


Operation  3:  List  all  possible  types  of  trajectories  in  the  state  space 

by  writing  down  all  possible  sequences  of  states  leaving  the  boundary  and 
delete  redundant  sequences  by  using  the  following  criterion:  Consider  every 
pair  of  sets  of  states  I  and  I'  satisfying  I  c  J'  and  max(J)  =  max(J'). 
every  trajectory  where  at  least  one  state  x^  €  J' /I  leaves  the  boundary 
(backwards  in  time)  after  all  states  of  I  have  left  the  boundary  is  redund¬ 
ant  and  can  be  deleted. 


Illustration:  For  the  network  of  Figure  2  the  list  of  all  possible  trajector-. 
ies  appears  in  Table  2  and  all  trajectories  except  9),  10)  and  13)  are 
redundant.  For  example,  sequences  12)  and  8)  are  redundant  with  J  «■  {x2»Xj} 
and  I’  =  {x^^.x^}  (the  notation  is  as  in  Operation  3),  sequences  1),  2), 

3)  are  redundant  with  I  -  {x^},  I'  =  {x^^}  and  similarly  for  the  other 
sequences. 

Motivation:  To  justify  the  redundancy  criterion,  consider  the  trajectory 

■^1*^2*  ‘  *  *  "  ’  ’^n*  *  * '  *  where  denotes  the  set  of  states  that  are 

away  from  the  boundary  x=0  during  segment  i  (backwards  in  time).  Since 
we  consider  only  trajectories  with  states  that  do  not  return  to  the  boundary. 

Now  assume  that  the  above  trajectory  satisfies 
the  criterion  in  Operation  3  and  with  the  notations  of  Operation  3,  let  m,  n 
be  the  smallest  indices  satisfying  J  c  and  x^  €  Jn  respectively. 

By  the  assumption  in  Operation  3  we  have  m  >  n.  We  show  that  the  above  trajectory 
is  redundant  by  producing  another  trajectory 

U  Xj), 

such  that  all  feedback  control  regions  constructed  by  the  first  trajectory 
are  also  constructed  by  the  second  (note  that  the  only  difference  between  the 
trajectories  is  the  segment  when  Xj  leaves  the  boundary). 


we  have  1^  a  1^  <z  . . . 
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Observe  first  that,  again  with  the  notations  of  Operation  3,  we  have 

max  (I  U  xp  *  max(If  U  x^)  ■  aax(r')  =  max(I) 

where  the  equalities  follow  from  Theorem  2c),  the  fact  x^  €  J'  and 
max(I’)  ■  max(I).  This  implies,  again  by  Theorem  2c),  that 


max (I  .  U  x.)  -  max (I  U  x.  U  (I  ./I))  =  max(I  U  (I  ./I))  ■  max(I  .). 
n-i  3  j  n-i  n-i  n-i 

Consequently,  ^  and  In  ^  U  x^  satisfy  the  conditions  of  Theorem  2a), 
i.e.  all  extremal  solutions  of  MFP(J  ,)  are  also  extremal  solutions  of 
MFP(Jn  ^  U  Xj).  Moreover,  by  Theorem  3,  the  costate  functions  are  identical 
for  both  trajectories,  implying  that  the  feedback  control  region  resulting 
from  the  segment  corresponding  to  Jn_^,  in  the  first  trajectory  results  also 
from  the  segment  corresponding  to  Inl  U  x^  in  the  second. 

Since  the  remaining  segments  of  both  trajectories  are  the  same,  the  resulting 
feedback  control  regions  are  identical  and  hence  the  first  trajectory  is 
redundant . 


Operation  4:  For  every  remaining  trajectory  from  the  list  of  Operation  3, 

and  starting  with  value  zero  for  all  costates  carry  out  the  following  steps 
for  every  segment  of  the  trajectory,  starting  at  the  first  segment: 

(a)  Set  A'  «  \i  ♦  1  Vxj  €  I. 


(b)  From  among  all  solutions  {y*}  of  MFP(J)  obtained  in  Operation  2  choose 
those  that  maximize  the  expression 

*  Vi 


(c)  Every  solution  obtained  in  (b)  determines  a  ray  in  the  state  space  and 


a  corresponding  set  of  controls.  Let  V  be  the  set  of  rays  determined 
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by  all  these  solutions  and  construct  the  region 


Vl  *  Co(Rp  U  ^/Rp 


where  ^  is  the  feedback  control  region  obtained  from  the  current 
step,  Rp  is  the  feedback  control  region  obtained  in  the  preceding 
step  (i.e.  from  the  previous  segment  of  the  trajectory)  and  CQ  denotes 
convex  hull  (see  [13,  p«  175]).  In  the  constructed  feedback  control  region  any 
control  from  the  abovementioned  set  that  does  not  take  the  state  outside 
of  the  region  is  optimal. 


Illustration:  In  our  three  dimensional  example  of  Figure  2,  in  order  to 

illustrate  the  constructed  feedback  control  regions  (that  are  convex  poly¬ 
hedral  cones  with  vertex  at  the  origin  [2,  Thm.  3]  it  is  easiest  to  draw 
a  plane  defined  by  the  points  with  coordinates  (0,0,1),  (0,1,0),  (1,0  0) 

(see  Fig.  4).  The  remaining  trajectories  of  Operation  3  are  9),  10),  13) 
of  Table  2,  and  the  costates  to  be  considered  according  to  Operation  4a) 
appear  in  Table  3.  The  control  that  maximize  the  expression  in  Operation  4b) 
are  given  in  Table  4.  Each  segment  of  each  trajectory  determines  a  feedback 
control  region  which  is  drawn  in  Figure  4  and  the  pair  appearing  in  the 
angular  parentheses  <  >  in  each  region  indicates  the  corresponding 
trajectory  and  segment  respectively.  For  example  in  trajectory  9,  the  first 
segment  generates  the  x^  axis  and  the  second  one  generates  the  cone 
determined  by  the  rays  (0,0,1),  (0,2/3, 2/3),  (2/3, 0,2/3),  excluding  the  x3 
axis.  The  corresponding  optimal  controls  appear  in  Table  4. 
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Motivation:  Since  we  consider  only  non-redundant  sequences  Theorem  3 

dictates  that  every  costate  is  identically  0  when  its  corresponding 
state  is  on  the  boundary  and  grows  with  slope  1  (backwards  in  time)  when  its 
state  is  away  from  the  boundary.  Consequently  the  costates  behave  as  in 
Operation  4a).  From  Theorem  lb)  follows  that  all  controls  optimizing  the 
expression  of  Operation  4b)  appear  in  the  lists  generated  by  Operation  2 
and  consequently  it  is  enough  to  check  the  value  of  this  expression  and 
select  the  controls  providing  the  maximal  value. 


IV.  CONCLUSIONS 

In  this  paper  we  presented  an  approach  leading  to  the  implementation  of 
the  algorithm  suggested  in  [1],  [2]  for  finding  a  feedback  solution  to  the 
problem  of  dynamic  routing  in  networks.  The  linear  programs  arising  from 
the  necessary  and  sufficient  conditions  for  optimality  were  transformed  into 
maximal-flow- in-network  problems  and  several  properties  concerning  theiT 
solutions  were  obtained.  The  properties  lead  to  the  development  of  an 
implement able  algorithm  for  building  the  feedback  space  for  the  problem  of 
dynamic  routing  in  single  destination  networks  with  zero  inputs  and  when  the 
cost  functional  has  no  priorities.  Two  inherent  features  of  the  problem 
limit  the  maximal  size  of  the  networks  for  which  the  algorithm  is  applicable 
under  moderate  computational  resources.  The  first  is  the  need  of  considering 


a  great  amount  of  piecewise  linear  trajectories  in  state  space  to  assure 
complete  covering  of  the  feedback  space.  The  second  is  the  need  for  finding 
all  solutions  to  the  linear  programs  defined  by  the  above  trajectories  to 
obtain  the  optimal  controls.  Taking  advantage  of  the  degenerancy  of  the 
linear  programs,  the  algorithm  severely  reduces  the  number  of  trajectories 
that  have  to  be  considered.  In  the  example  of  Section  III  only  3  out  of  13 
trajectories  had  to  be  considered.  In  [4]  an  example  of  a  network  with  four 
states  is  presented  in  which  only  18  out  of  75  trajectories  have  to  be 
considered.  Moreover,  a  simple  method  for  finding  all  solutions  of  the  linear 
programs  was  developed.  The  method  is  based  on  pivoting  operations  on  net¬ 
works  and  in  [4]  several  examples  are  presented  when  the  largest  one  consists 
of  a  five  states  network  that  lead  to  184  different  solutions  and  represent¬ 
ations  with  a  reasonable  CPU  time  (7.41. sec.  on  an  IBM  370/168  computer). 
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APPENDIX 

The  Method  for  Finding  all  Solutions  to  the  Linear  Programs 

The  method  suggested  here  is  based  on  techniques  of  pivot ing-on-networks. 
The  concepts  used  here  can  be  found  in  [12].  Recall  that  Operation  2  of  the 
algorithm  dictates  that  all  optimal  solutions  have  to  be  found  only  for  those 
MFP(J)'s  for  which  there  is  no  set  I*  =>  I  satisfying  max(Jf)  =  max(J). 

At  maximal  flow,  the  corresponding  networks  N(J)  can  be  reduced  to  networks 
containing  only  the  nodes  in  X,  when  (X,X)  is  the  minimal  cut  close  to  the 
source  (see  Theorem  2b)).  For  a  reduced  network  JVr(J)  the  first  (extended) 
basic  optimal  solution  is  given  by: 

yi  3  bi  Vxi  £  *r(I) 

uik  =  0  Vxi*xk  6  0  S(I)  * 

The  links  y^  are  taken  to  be  basic  and  the  remaining  links  non-basic. 
Starting  at  these  first  optimal  extended  basic  solution  the  method  picks 
a  non-basic  link  and  looks  foT  a  cycle  formed  by  basic  links  and  the 
considered  non-basic  link.  The  existence  of  such  a  (unique)  cycle  is 
ensured  since  every  extended  basic  solution  corresponds  to  a  spanning  tree 
in  the  network.  A  pivot  is  performed  on  the  cycle  reaching  a  new  extremal 
solution  ot,  if  the  cycle  is  degenerate,  another  representation  of  the 
original  extremal  solution  (i.e.,  the  non-basic  link  is  declared  basic,  and 
a  basic  link  becomes  non-basic  while  the  flow  on  the  links  of  the  cycle  does 
not  change).  Every  new  solution  or  representation  is  numbered  and  kept  in 
memory  (if  it  does  not  exist  there  already).  The  above  process  is  performed 
for  all  non-basic  links  corresponding  to  every  solution  and/or  representation 


in  memory  until  no  new  solution  or  representation  can  be  reached.  This 
ensures  that  at  the  end  of  the  process  all  optimal  solutions  had  been  found. 
Finally,  only  one  representation  per  extremal  solution  is  kept. 

Note  that  from  two  different  representations  of  an  extremal  solution, 
different  new  extremal  solutions  can  be  reached,  therefore,  it  is  essential 
to  consider  all  the  representations  of  the  extremal  solutions  in  the  process. 
Unfortunately,  the  number  of  solutions  and  representations  is  not  known  a 
priori;  only  an  upper  bound  is  known  since  every  basic  solution  corresponds 
to  a  spanning  tree  of  the  network  (the  number  of  spanning  trees  of  a  network 
can  be  calculated),  however,  not  every  spanning  tree  leads  to  a  feasible 
basic  solution.  Therefore,  in  practice  this  number  is  meaningless. 


1 
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u12  U21  u23  uld  U3d 


y3  u12  U21  u23  uld  U3d 


2  0  0  0  0  2 


Table  1  -  Subnetworks  and  their  different 
solutions 
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1) 

X1 

X2 

X3 

2) 

X1 

X3 

X2 

3) 

X1 

x2'x3 

4) 

x2 

X1 

x3 

5) 

x2 

X3 

X1 

6) 

X2 

X1'X3 

7) 

x3 

X1 

X2 

8) 

X3 

X2 

X1 

9) 

x3 

x1,x2 

10) 

Xj  ,x2 

X3 

11) 

X1’X3 

x2 

12) 

x2*x3 

X1 

13) 

i ^2 9 Xj 

Table  2  -  List  of  trajectories  of  states 
leaving  the  boundary 


Trajectory  (from  Table  2) 
9 

10 

13 


First  segment 
(0,0,1) 
(1,1,0) 
(1.1.1) 


Second  segment 
(1.1.2) 
(2,2,1) 


Table  3  -  Costate  values  X  =  (Xj,X2,Xj) 


Traj  ectory 
9 

10 

13 


First  segment 
c)  i) 

b)  i),  ii),  iii) ,  iv) 
a)  i)  -  viii) 


Second  segment 

a)  i),  ii),  v),  vii) 

a)  iii),  iv),  vi) ,  viii) 


Table  4  -  Controls  maximizing  expression  appearing 
in  Operation  4b)  (entries  correspond  to 
controls  determined  in  Table  1) 


